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ABSTRACT 


Sequential  decoding  with  direct  quantization  of  the  M  orthogonal  channel 
outputs  is  investigated.  ^comp’  the  ra^e  below  which  sequential  decoding 
computations  are  manageable,  is  computed  for  a  memoryless,  coherent, 
gaussian  channel.  These  computations  show  the  direct  quantization  to  be 
more  effective  than  the  list-of-  JL  quantization,  when  sufficient  storage  is 
available  in  the  decoding  device.  The  performance  of  a  continuous  output 
channel  can  be  approached  by  quantizing  each  output  to  only  a  few  bits.  The 
direct  quantization  of  channel  outputs  also  simplifies  the  cumbersome  com¬ 
putations  which  would  result  if  list-of-  JL  decoding  were  implemented  in  a 
general  purpose  computer. 
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M-ary  Channel  Quantization  for  Sequential  Decoding 


An  M-ary  modulation  scheme  has  been  used  in  conjunction  with 

1  2  3 

sequential  decoding  in  the  Lincoln  Experimental  Terminal  *  (LET).  In 

the  M-ary  modulation  one  out  of  M  orthogonal  waveforms  is  transmitted 

during  one  transmission  period.  The  receiver  then  tries  to  decide  which 

waveform  (message)  was  transmitted  by  observing  the  signal  amplitudes 

corresponding  to  the  M  possibilities.  When  M-ary  modulation  is  used  with 

sequential  decoding,  the  communications  capacity  (for  a  given  transmitted 

power)  can  be  increased  by  either  making  M  large  or  quantizing  the  outputs 

of  the  M  orthogonal  channels  more  precisely.  Increasing  M  implies  a 

more  complicated  system  and  requires  more  bandwidth,  since  the  receiving 

equipment  is  essentially  M  receivers,  each  one  matched  to  one  of  the  M 

transmitted  orthogonal  waveforms. 

On  the  list-of-£  quantization  system,  the  M  orthogonal  channel  outputs 
are  compared  and  the  receiver  designations  stored  in  a  list  in  decreasing 
order  of  output  amplitude.  Only  the  highest  J l  channels  are  recorded  in  the 
list.  This  requires  storing  l  identification  numbers,  each  number  corres¬ 
ponding  to  one  channel.  The  decoder  then  uses  the  list  position  of  the 
channel  to  decide  what  was  transmitted. 

If  the  individual  orthogonal  channel  output  amplitudes  were  quantized 
directly  with  some  precision,  different  channel  performance  would  be  obtained. 
The  remainder  of  this  note  deals  with  such  an  investigation. 

The  expression  for  R  (the  rate  below  which  one  can  decode 

r  comp 

sequentially  without  excessive  computation  requirements)  was  derived  for  a 
directly  quantized  memoryless  channel.  The  equation  was  evaluated  for 
coherent  reception  over  a  channel  with  gaussian  noise.  Comparisons  of 
these  results  with  list-of-X  quantization  are  encouraging.  The  direct  com¬ 
putation  has  a  higher  absolute  potential  and  even  provides  a  higher  Rcomp 
for  given  storage  requirements  in  some  interesting  cases.  It  appears  that 
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one  may  want  to  quantize  the  orthogonal  channel  outputs  to  as  low  as  two  or 
three  bits  and  would  never  want  to  quantize  signals  to  more  than  four  bits. 

The  direct  quantization  has  the  additional  advantage  of  conceptual 
simplicity  as  well  as  being  more  attractive  to  use  if  sequential  decoding  is 
performed  in  a  general  purpose  digital  computer. 

Computation  of  X^ 

At  the  transmitter  end  we  have  a  stream  of  bits  coming  from  an  informa¬ 
tion  source.  These  bits  are  fed  into  a  shift  register  which  has  parity  net¬ 
works  connected  to  it.  Several  parity  bits  are  computed  and  added  to  the 
source  bit  stream.  Then  log^M  of  the  bits  are  grouped  together  and  the 
modulator  selects  and  transmits  one  of  M  orthogonal  signals.  Although  not 
necessary,  we  will  assume  for  illustration  purposes  that  the  log^M  bits  con¬ 
sist  of  one  information  bit  and  log^M-  1  parity  bits.  For  example,  if  M  =  16, 
one  information  bit  and  three  parity  bits  would  be  used  to  select  an  orthogonal 
waveform.  The  results  obtained,  however,  are  quite  general  and  can  be  used 
for  other  system  configurations . 

The  receiver  assumes  that  up  to  a  particular  time  it  has  decoded  the 

received  signals  into  a  correct  sequence  of  bits.  It  uses  these  bits  in  a 

shift  register  and  parity  networks  identical  to  the  ones  at  the  transmitter 

and  tries  either  a  one  or  a  zero  for  the  next  bit.  Two  sets  of  log^M  bits  are 

obtained,  one  for  the  case  when  a  one  was  hypothesized  for  the  next  digit,  and 

the  other  when  a  zero  was  hypothesized.  Now  a  generalized  distance  metric 

X.,  which  is  a  monotonic  function  of  the  probability  that  the  hypothesized 

digit  was  transmitted  given  that  the  particular  channel  outputs  were  received 

is  computed  for  both  hypotheses  and  the  hypothesis  with  the  higher  X^  is 

assumed  tentatively  to  be  the  transmitted  bit.  A  running  sum  of  the  X  ^  is 

kept  which  hopefully  increases  consistently  as  successive  bits  are  decoded. 

If  the  sum  decreases  sufficiently  a  mistake  might  have  been  made  somewhere 

along  the  way  and  the  process  is  backed  up  and  different  sequences  are  tried. 

4 

This  sequential  decoding  process,  due  to  Fano,  searches  for  a  sequence  which 
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will  eventually  give  the  highest  sum  of  X 

Let  the  hypothesized  transmitted  message  be  designated  by  (one  of 
the  M  orthogonal  transmitted  signals),  and  let  y  be  the  vector  of  outputs  of 
the  M  orthogonal  channels  during  the  i-th  transmission  of  the  channel.  We 
compute  the  a  posteriori  probability  that  was  transmitted  given  that  y. 
was  received.  For  a  sequence  of  n- 1  bits  the  a  posteriori  probability  of  the 
decoder  picking  the  transmitted  sequence  of  bits  is  the  product  of  the  individual 
bit  a  posteriori  probabilities,  if  the  channel  is  memoryless  and  the  noise  is 
white  gaussian.  The  object  of  the  decoder  then  is  to  maximize  this  a  posteriori 
probability  or  any  monotonic  function  of  it.  The  logarithmic  function  of  this 
probability  allows  simple  calculations. 


n-  1  *  _ 

n-  1 

L  [  log  p(x.'  |y  )  -  U]  = 

i=  0  c 

i=  0 

(1) 

where  x*  is  the  i-th  hypothesized  transmitted  symbol,  y^  is  the  i-th  received 
vector  of  channel  outputs,  L^  is  the  measure  of  correctness  up  to  the  n-th 
decision  and  U  is  simply  a  constant  selected  so  that  L^  increases  for  correct 
sequences  of  decoded  bits  and  decreases  for  incorrect  sequences.  The  exact 
value  of  U  has  been  shown  not  to  affect  the  number  of  decoding  computations 
greatly.  ^ 

In  a  list-of-  4  type  demodulation  system  the  outputs  of  the  M  orthogonal 
channels  are  compared  with  one  another,  and  a  list  is  kept  of  the  4  channel 
identification  numbers  with  the  highest  output  signals.  A  table  of  values  of 
X  (called  a  metric)  is  kept  which  assigns  a  value  for  X  to  the  various  channels 
depending  on  what  position  in  the  list  the  channel  occupied.  This  method  loses 
some  information,  since  the  metric  is  computed  for  the  average  channel  con¬ 
ditions.  The  individual  orthogonal  channel  amplitudes  are  not  utilized,  only 
the  relative  positions  on  the  list  count.  The  following  paragraphs  describe 
how  the  individual  channel  output  amplitudes  can  be  used  in  the  calculations 
if  the  individual  channel  outputs  are  quantized  directly  and  stored  in  the 
decoding  computer. 
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The  X  .  can  be  computed  directly  when  the  individual  channel  outputs 

'Jc  — » 

are  available.  We  can  compute  p(x^  |y^)  by  Bayes  rule: 

P(xi  )P  (Yi  Ixi  ) 


P(xi  Iy^ 


p(yj 


(2) 


When  the  k-th  orthogonal  waveform  is  transmitted,  the  k-th  orthogonal 
channel  output  will  be  a  random  variable  with  an  amplitude  probability  density 
p  (y,  )  due  to  transmitted  signal  and  noise,  and  the  other  channel  outputs 
will  have  an  amplitude  probability  density  Pn(Yj)  due  to  noise  only.  These 
densities  are  independent  of  i. 

Given  that  the  k-th  signal  (x^)  was  transmitted  during  the  i-th  interval, 
we  have  the  conditional  probability 


M 

P^iKk*  =  Ps(yik)  ,n,  Pn(yij> 

J=  1 


(3) 


Since  the  individual  noise  components  are  statistically  independent,  the 
probability  that  a  particular  y  occurs  is 


p(yi) 


M  M 

L  plx^)  ps(yh)  n  P„  (Yi)  , 

h=  1  j=  1 


(4) 


where  p(x.  )  is  the  probability  that  the  h-th  message  was  transmitted  during 
an  interval  and  is  assumed  to  be  a  constant  l/M.  Using  Eq.  (3)  and  (4)  in 
Eq.  (2)  we  have 


p<x^  W 


ps  (yik)/pn  (yik> 

1 

L  Ps (yih)^Pn(yih) 


(5) 


h=  1 


where  p(x.,  |y.)  is  the  probability  that  the  k-th  message  was  transmitted 

during  i-th  transmission  interval  given  that  y- was  received.  Then  the 
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measure  of  correctness  of  the  i-th  decision  is  given  by 

M 

Xi  =  1°g2lPs'yik)/pn(yik)l  -lo«2  I  ,2.  Ps^ih'/Pn^ih1'  '  U’ (6) 

h=  1 

if  the  k-th  message  was  hypothesized. 

The  sequential  decoding  algorithm  requires  the  computation  of  X  ^ 
every  time  the  decoder  tries  to  decide  on  a  bit;  hence,  the  computer  must  be 
able  to  compute  X ^  very  quickly. 

If  one  computed  X^  by  the  list-of-4  technique,  one  would  have  to  search 
through  the  JL  channel  identification  numbers,  find  the  position  of  the  desired 
message  and  then  look  up  the  X^  which  corresponds  to  that  position.  While 
table  look  up  is  easy  in  a  general  purpose  digital  computer,  finding  a  particular 
number  in  a  list  is  not.  Consequently  list-of-^  decoding  is  a  very  cumber¬ 
some,  time  consuming  operation  on  a  general  purpose  digital  computer.  In 
addition,  getting  the  list  itself  is  not  a  trivial  matter;  indeed,  in  the  LET 
system  the  list  is  formed  by  analog  circuitry^  external  to  the  special  purpose 
digital  decoding  machine. 

Equation  (6)  shows  the  operations  required  to  obtain  the  metric  with 

direct  quantization.  Note  that  the  second  term  of  (6)  is  independent  of  k, 

thus  has  to  be  computed  only  once  no  matter  what  k  is  hypothesized.  To 

compute  this  term,  one  would  look  up  and  sum  the  ratio  of  the  probabilities 

and  then  compute  the  logarithm  of  the  sum.  The  frequently  computed  first 

term  of  (6)  can  be  obtained  by  a  simple  table  look  up.  The  tables  for  both  of 

these  computations  can  be  constructed  beforehand,  or  the  statistics  p  (y ) 

s  lx 

and  p  (y.,  )  can  be  updated  periodically,  thus  adjusting  the  metric  com- 
n  ix 

putations  to  the  most  recently  observed  channel  characteristics. 

R  of  Quantized  System 

comp 

— ♦ 

In  order  to  compute  X^,  we  need  to  have  the  values  of  y^  stored  in  some 
fashion.  One  way  to  accomplish  this  is  to  quantize  the  output  of  each 
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orthogonal  channel  and  store  the  quantized  values  in  a  digital  computer 

memory.  We  will  investigate  the  quantization  requirements  by  computing 

R  (the  rate  below  which  one  can  sequentially  decode  without  excessive 

comp  ^  7 

computation  requirement)  for  various  quantization  step  sizes  and  limits  of 
quantizations  . 

We  need  to  compute  R  for  a  discrete,  memoryless  channel  with 

comp  7 

M  channel  inputs  and  J  possible  channel  outputs.  The  output  can  be  visualized 
as  a  vector  y  with  M  components  (y  ^ ,  y3  ,  .  .  .  ,  y^.) .  Each  component  is 
quantized  into  N  intervals  so  that  the  channel  output  y  can  take  on 
different  values  (J  =  N^). 

7 

Reiffen  has  shown  that  R  for  a  discrete,  memoryless  channel 

comp  7 

such  as  ours  is  given  by 

max  |  J  r  M  , — — -  ~|2  \ 

'comp  =  lP<*k>J  J  '  l°g2  ^1  p(V  '/P'yj'V  J  j  ■  (7» 

where  p(x^)  is  the  probability  of  the  k-th  message  (x^)  being  transmitted 

and  p(y .  lx,  )  is  the  transition  probability  of  y.  being  received  given  that  x, 

J  ^  J  ^ 

was  transmitted.  The  value  of  the  transition  probability  is  given  by 

M 

plyj  IV  =  Ps(V  n  ,  ’  <81 

J  m  =  1 

m^k 

where  p  (y,  )  is  the  discrete  probability  density  function  of  the  channel  output 

S  rC 

of  k-th  channel  (i.e.,  the  channel  containing  the  signal)  and  Pn(Ym)  is  the 
discrete  probability  density  function  of  the  other  channels  (i.e.  ,  channels 
with  noise  only). 

Substitution  of  (8)  into  (7)  yields  after  simplification  and  summing 
probabilities  to  1 


(9) 


max  i 

M  1 

rN 

-|Z  M 

M  j 

Rcomp  =  fP(xk)j  |  ”  lo^2  < 

'  (XH 

* 

Ln=  1 

*/p  (y  )p  (y  ) 

n  ’ n  's  n 

1  L' 

J  k=  1 

gt'i  P<Xk,P(Xg))  j 

g^k 
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where  Pg(yn)  is  the  probability  that  the  orthogonal  channel  output  with  the 
signal  falls  into  the  n-th  quantization  interval  and  Pn(Yn)  is  the  probability 
that  the  orthogonal  channel  with  noise  only  falls  into  the  n-th  quantization 
interval.  Equation  (9)  is  seen  to  be  maximized  when  p(x^)  is  a  constant 
l/M.  Thus 


R 

comp 


-  log2 


J_  M- 1 
M  +  M 


N 

L 

_  n=  1 


(10) 


Re  suit  s 

Equation  (10)  was  evaluated  for  a  coherent  gaussian  channel  whose 


and 


■(y~a)‘ 


ps(y) 


1  2 
e 


,  ,  1  2 

pn,y)  =  —  e 


(ID 


(12) 


where  p  (y)  and  p  (y)  are  the  continuous  probability  densities  for  the  signal 

S  n  IZE~ 

•  r 

and  noise  respectively  and  a  =  —  ,  in  which  E^  is  the  received  energy 

o 

when  an  orthogonal  waveform  is  transmitted  and  Nq  is  the  single- sided  white 
gaussian  noise  power  density. 

The  output  space  y  is  partitioned  into  N  increments  and  the  values  of 

Ps(yn)  and  PR(yn)  are  simply  integrals  of  (11)  and  (12)  over  the  n-th  increment. 

For  all  calculations,  equal  increments  were  assumed  between  some  minimum 

value  y  .  and  maximum  value  y  ,  which  correspond  to  the  minimum  and 

maximum  channel  output  voltages.  The  lower  end  increment  and  the  upper 

end  increment  were  made  to  include  all  values  of  y  less  then  y  and  all 

values  of  y  more  than  y  respectively.  The  values  of  y  .  and  y 

7  7  max  r  7  7  min  7  max 

were  optimized  so  that  R  was  maximized.  Figures  1-4  show  R 

^  comp  &  comp 
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as  a  function  of  E  /N  for  N  =  Z,  4,  8,  16  and  different  values  of  M.  The 

r  o 

figures  also  show  the  continuous  case,  which  would  result  if  the  channel 

outputs  were  quantized  into  an  infinite  number  of  increments.  This  is  the 

best  that  sequential  decoding  could  do  with  an  M-ary  channel.  The  figures 

show  the  list-of-£  R  for  different  values  of  £  as  well. 

comp 

It  is  obvious  that  the  direct  quantization  of  signals  has  a  higher  absolute 

potential  than  the  list-of-4  quantization.  Figure  5  shows  signal-to-noise 

ratios  that  are  required  to  obtain  a  typical  R  with  a  given  available 

7  r  comp  b 

storage.  One  can  see  that  the  direct  quantizer  requires  a  lower  signal-to- 

noise  ratio  only  when  large  amount  of  storage  is  available.  If  storage 

availability  is  no  problem  it  appears  that  N  =  8  or  16  might  be  as  fine  a 

quantization  that  one  would  use.  An  interesting  fact  is  that  the  direct 

quantization  is  more  efficient  when  low  values  of  R  are  required. 

comp  ^ 

The  optimum  quantization  levels  are  shown  in  Fig.  6.  As  shown  by 

Eq.  (10)  the  levels  are  independent  of  M,  so  curves  of  optimum  quantization 

region  are  shown  as  a  function  of  E^/N^  for  values  of  N.  Since  the  signal 

and  noise  distributions  are  symmetrical,  y  =  a  -  y  is  the  proper 

7  7  max  7  mm  r  r 

choice.  The  value  of  y  and  y  is  seen  not  to  be  a  very  critical  one. 

7  mm  7  max  7 

Figure  7  shows  the  plots  of  R  as  a  function  of  the  limits,  which  indicates 

r  comp 

that  the  maximum  is  quite  broad  and  not  much  is  lost  at  other  signal-to-noise 
ratios  if  the  limiting  and  quantization  are  picked  optimum  for  the  minimum 
expected  signal-to-noise  ratio.  The  curves,  however,  assume  that  the 
probability  densities  which  are  used  in  the  computer  are  the  correct  ones. 

If  the  probabilities  change  and  the  computer  does  not  change  the  metric 
calculation  tables  the  quantization  interval  is  somewhat  more  sensitive  to 
channel  variations. 

Conclusion 

The  direct  quantization  of  every  orthogonal  channel  output  to  a  modest 

number  of  bits  (3  or  4  bits)  provides  R  very  close  to  the  best  that  can 

r  comp  7 

be  accomplished  over  an  M-ary  channel.  In  comparison,  this  is  substantially 

>Jc 

The  list-of- £  curves  are  taken  from  K.  Jordan's  work  (Ref.  5)  which  has 
guided  the  present  study  in  all  phases. 
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better  than  list- of- i  quantization .  When  the  direct  quantization  is  compared 

to  list-of-4  quantization  assuming  the  same  memory  storage,  the  direct 

quantization  is  better  for  more  complicated  systems,  that  is  systems  with 

larger  memories.  The  list-of-£  provides  higher  R  for  very  limited 

comp  7 

memory  machines.  The  computation  of  for  direct  quantized  system  is 
also  easier  to  perform  on  the  general  purpose  digital  computer. 
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Fig.  1.  R  for  M  =  4. 

°  comp 
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Fig.  2. 


R 


comp 


for  M  = 
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1  1 
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Fig.  3.  R  for  M  =  16. 
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Fig.  4.  R  for  M  =  32. 

°  comp 
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NUMBER  OF  BITS 

Fie.  5.  E  /N  required  for  given  R  as  a  function 

r  o  ^  comp 

of  number  of  storage  bits  M  =  16. 
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min  mox 


Fig.  6.  Optimum  quantization 
limiting. 
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7.  Effect  of  limiting 
comp* 
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